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Measurement Services Association 
Questionnaire 

At- the_ 1986 annual, meeting of the Measurement Services 
Association in San Francisco , it became apparent that many of us 
were interested in what our colleagues were doing in the areas of 
hardware_ and_ sof tware innovation . It was felt by many that a 
questionnaire which looked into hardware and software used by 
testing centers throughout the country could be beneficial in two 
ways: _ i) __ Results from such a questionnaire could allow 
individual _centers to assess their own position in relation to 
other centers^ and 2) individual centers could be made aware of 
available software and where to obtain it in order to improve 
their operations. 

With this_ in mind^ in the intervening twelve months since our 
last visit f and under the direction of MSA Chairman Rod Gillis, a 
questionnaire was prepared and sent out to members of the 
Association^ A total of _211 questionnaires were sent. Sixty- 
four ^ were_ returned for analysis. The purpose of this report is 
to ...<3isseminate_the information thus gathered so as to enable MSA 
members to accomplish the dual goals listed above. 

Be aware that the purpose of this questionnaire is to provide a 
vehicle_wherein testing center personnel can become familiar with 

software availability. Neither MSA, it's chairman, nor editor , 

make^ any recommendations as to suitability of software for a 
par ticuiar institution or even if specific software will perform 

as indicated. Software transportability is a very "site 

specif Ic'^jDrcbi em and what works well in one area may not work at 
ail in another. If you are interested in upgrading software in 
your center, you would be best advised to personally contact the 
institution indicated _ and deal directly with personnel there in 
order to consider software exchanges or purchases. 

PART j - HARDWARE €QNS IDEaATIONS . The first part of the 
questionnaire dealt with specific hardware used within our 
offices. We were not only concerned with which computers and 
scanners were in use L but also with the percentage of usage on 
each piece of equipment. 

Computer Usage . F±fjy-tw6 of the sixty-two respondents ( 84% ) who 
replied to this section _indicated usage of a mainframe computer 
to some extent (the replies were literally from 1% to 100%). 
Fifty of the respondents ( 81% ) indicated some use of a 
microcomputer. Only fourteen of the schools (23%) utilize a 
minicomputer in their testing operations. 

Ten institutions use a jnainframe exclusively , while eight of the 
surveys indicated 100% reliance on microcomputers only." Only one 
reply indicated 100% reliance on a minicomputer. 
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Forty-seven of the fifty surveys that mentioned microcomputer 
tisane indicated ase of IBM-compatible equipment . Forty of the 
forty-seven used : only IBM- compatible equipment. Only three 
respondents utilize noh- IBM-compa t i bl e microcomputers 
exclusively,. Seven respondents have ftpple_ machines and six 
others mentioned DEC, TRS, Burroughs, Commodore, Wang, and Altos 
machines. The distribution of usage oh computers is shown in 
Table 1. 

Table 1 



Type / %Usaqe 


<1G% 


201 


36' 


h 40% 50% 


60% 


70% 


80% 


90% 


100% 


Mainframe 


23 




5 


6 8 


2 


2 


5 


li 


23 


Minicomputer 


3 


5 


2 


2 - 






3 


5 


3 


Microcomputer 


23 


5 


5 


5 6 


2 


6 


2 


8 


13 



n=62 , all figures shewn as percentages of n. 

Note, for example , 23% of respondents use a mainframe system up 
to 10% of the time r 5% up to 30% of the time, 6% up to 40% of the 
time, ... , 23% use their mainframe 100% of the time, etc. 

Optical Scanners. Just as IBM-compatible micros dominate that 
aspect of the market, so also NCS equipment dominates when a 
distribution of scanners is looked at. Forty-three of fifty-nine 
replies concerning scanners indicated exclusive use of NCS 
equipment . This represents 73% of the survey. Only 8 replies 
(14%) used all non-NCS equipment . Of the 8 surveys reporting 
usage of scanners by more than one manufacturer, 7 utilized an 
NCS scanner in addition to one or more other machines • Thus , 
fifty of fifty-nine institutions use NCS equipment to some 
rree. 



Five institutions use ScanTron equipment exclusively and four 
others use it in conjunction with other machines . One user 
reported use only of Cognitronics equipment and three others used 
Cognitronics scanners with other machines . No other scanner 
manufacturer was mentioned more than one time either alone or 
with another piece of equipment • 

Distribution of scanner usage is shown in Table 2. 

Table 2 
Scanner Utilizations 



Type / %Usaqe 


<10% 


20% 


30% 


40% 


50% 


60% 


70% 


80% 


90% 


100' 


NCS 2050 






















NCS 3000 


11 


2 














2 


12 


7000,01,03 






2 












4 


12 


7005,08,10,18 


2 




5 


5 




2 




2 


2 


34 


Scantron 




2 














2 


4 


IBM 1230 






2 
















Cognitronics 


2 








2 






2 




2 


Other 


2 






2 


2 


.4 


2 









n=59 , all figures shown as percentages of n. 
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PART II - SOFTWARE CONSIDERATIONS > The second part of bur survey 
dealt with software being used by the various centers represented 
by membership in MSA. We wanted to look at classroom test 
scoring programs which we assumed were in use in almost all 
centers (we were right) as well as other programs such as grade 
roils; item banking, psychological and vocational interest 
testing, placement testing, and other, perhaps more specialized 
programs. 

Class ro caa—Teg^— Scoring Software . Of the sixty-four surveys 
returned for analysis , fifty-five indicated the use of a 
classroom test scoring program. Table 3 shows the percentage of 
users with the features they have in their programs and also how 
important those features are felt to be. In the columns 
indicating importance, the figures may not total to 100% because 
many did not make an indication of their feelings in that area. 

Table 3 

Classroom Test Scoring 



feature- 



4iav© 



Importance 



Item Analysis 85 

Alpha list of names 75 

ID list of scores 84 

Score distribution table 69 

Score data written to disk 65 

Rawdata written_ to disk 64 

Subscores available 58 

Differential weighting 45 

Reliability coefficients 78 

Collusion Coefficients 5 

Output plot or graph 55 
Ind ividual s core report — 35 



Essential 



Useful 



Not Useful 



80 

73 
65 
60 
49 
38 
44 
40 
65 
4 
33 

-id 



7 
^5 
24 
11 
24 
31 
25 
16 
13 
15 
33 
22 



4 
4 
4 
4 
4 
5 
5 

11 
4 
5 
5 

11 



n=55, all figures shown as percentages of n. 

Of just about as rnuch inter est as the programs in use, is the 
equipment those programs run on and the source of the software. 
Thirty-five percent _ of _ those responding indicated use of a 
mainframe to >_rjun_ their program while twenty-four indicated use of 
a micro. Five percent used a mini and 36% didn't indicate what 
they_ used. __ Oyer half of the respondents failed to indicate the 
scanner used for their ^ classroom testing program, but given '-he 
distribution of scanners discussed above, undoubtedly most of the 
programs use NCS equipment. 

Almost two-thirds cf ; the respondents indicated the software used 
was developed in-house while only five percent indicated it had 
been purchased commercially. Eleven percent received their 
software from another institution. Table 3A shows these 
distributions . 
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Table. .3 A * 
Hardware Configurations 



Computer Used: 




Scanner Used: 




Mainframes 


35% 


NCS 3000 


5% 


Minicomputers 


5 


NCS 7000-7Q03 


13 


Microcomputers 


24 


NCS 7005-7018 


11 


No indication 


36 


Cognitronics 


5 






Scantron 


2 


Source: 




Other . 


_7 


Commercial 


5% 


No indication 


56 


Developed in-house 


64 






Another institution 11 






No indication 


20 






*Figures may not total 


100% due 


to rounding error 




Eighty-five percent of 


the resp 


ondents indicated they were 


happy 


with their software. 







Psychological/Vocational Interest Software Only twenty-one 
surveys reported back as to psychological and vocational interest 
software being used. This would apparently indicate not nearly 
so many centers process this type of information as do normal 
classroom scoring. Most of those responding indicated they were 
scoring the Strong -Campbell Interest* Inventory. _ Tests receiving 
a significant number of "votes" are shown in table 4. 

Table 4 

Psychological/Vocational Interest Programs 



Test Processed 



Percent scoring this test 



Strong-Campbell_ISCII) 57 
Myers-Briggs (MBTI) 43 
Minnesota Multiphasic (MMPI) , 43 
Calif. Psychological (CPI) 29 
Edwards Personal Preference (EPPS) 14 
Career Assessment Inventory (CAI) 10 
Omnibus (OPI) 10 

Personal Orientation Inventory (POI) 1Q 

n=21, all figures shown as percentages of n. 

Other tests mentioned: Personality Research, Form/ Canf ield 
Learning Styles Inventory, 16PF, Tennessee, GATB , Runner Scale, 
Kentucky Comp. Listening Skills, Temp. & Values Inventory, Kuder , 
Herrmann, Bipolar, SSHA, Firo-B, Taylor-Johnson, Zung-D, STAI-X, 
STAI-Y, AUI, HWDYKY 

An interesting fact concerning this type of testing is that* for 
the most part , the software is micro-based, and a significant 
percentage of the software is commercial . Eighty-one percent of 
the programs plot scores and of this number , 12% "require a 
graphics printer. This information is tabulated in table 4A. 
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Table 4A* 



Hardware Configurations 
Psychological/Vocational Interest Programs 



Computer Used: 




Scanner Used: 




Mainframes 


29% 


NCS 3000 


19% 


Minicomputers 


10 


NCS 7000-7003 


5 


Microcomputers 


48 


NCS 7005-7018 


10 


No indication 


14 


Scaritron 


5 






Other 


5 


Sourcf: 




No indication 


52 


Commercial 


29 






Developed in-house 


52 


Plot results: 




Another institution 


5 


Yes 


81% 


NO indication 


14 


Graphics Printer 


12% 



Figures may not total 100% due to rounding error 



ft problem associated with this type of software : nvolves Federal 
copyright laws that protect the tests. Development of a scoring 
program using scales or items developed by another person or 
company can violate the copyright • Be aware that if you attempt 
to develop a program yourself, that a scoring license will 
probably be necessary to allow you to legally proceed. In some 
cases, the company or individual holding the copyright will not 
allow any software other than their own to be developed and no 
license will _ be issued. This isthecase, for example, with the 
MMPI and NCS. If you are to legally score the MMPI in your 
office, you must use the NCS software to do so. 

Other tests, such as the Strong-Campbell , are licensed out to 
third parties and privately developed software can be circulated 
so long as royalty payments are made to the publisher • _ In many 
cases, the royalty paid through, such a privately developed 
program is considerably less than that paid through use of the 
publisher's own software. 

Considering the small number of centers using this type of 
software, the overwhelming majority are pleased with their 
programs • Ninety-five percent of the users expressed 
satisfaction. 

Placement Testing Thirty-seven of the sixty-four surveys 
returned indicated processing of some type of placement testing. 
By far the largest single subject tested was mathematics. This 
was followed by English and then basic ski J ls_ programs of various 
types. This information is summarized in table 5. 
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Table 5 

_ : Placement_Testing Programs 

Test Processed Percent scoring this test 

Hath programs 89 
English programs 41 
Basic Skills Programs 35 
Foreign Language programs 19 
Chemistry Programs 19 

Reading programs . . 16 

n=37 f all figures shown as percentages of n. 

Other subjects mentioned: Computer Science, Nursing, Biology, 
Speech Teacher Certification, Journalism, Careers. 

Unlike psychological and vocational interest testing, most of the 
respondents in this area indicated their programs were mainframe- 
based and the larger NCS scanners were being used. By far the 
greatest percentage of programs were developed in-house and would 
therefore probably not be as transportable as other software 
mentioned in this report. This data is summarized in table 5A. 

Table 5A* 



Hardware Configurations 
P lacement Testing Programs 



Computer Used: 




Scanner Used: 




Mainframes 


35% 


NCS 3000 


8% 


Minicomputers 


3 


NCS 7000-7003 


5 


Microcomputers 


19 


NCS 7005-7018 


14 


No indication 


38 


Scantron 


5 






Cognitronics 


5 






Other 


3 


Source; 




No indication 


59 


Commercial 


5% 






Developed in-house 


78 


Plot scores: 




Another institution 


5 


?es 


14% 



No indication 11 
^Figures may not total 100% due to rounding error 

Almost three-fourth of the respondents ( 73% ) indicated 
satisfaction with their placement software • 

Specialized Testing Software In this category, we were looking 
for programs that handled specific tests that did not fit into 
the psychological/vocational interest, test category. The six 
tests mentioned most are listed in table 6. 
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Table 6 .. 
Specialized Testing Programs 



Test Processed 


Percent scoring^ this test 


ACT 


50 


GED 


25 


SAT 


18 


TOEFL 


14 


MAT 


14 


I TED 


11 



n=28, all figures shown as percentages of n. 



Other tests mentioned: ITBS, DAT, TASK, Teacher Evaluations , 
Teacher Certification and Licensing, CAT, Henmon-Nelson, SCAT, 
CEEB Achievement Tests, Const i tut ibhal Tests 

These tests seem to be more evenly split as to the type of 
equipment they run on. Software is available for all three types 
of computers and for. thei entire range of scanners. These 
programs are also mostly developed in-house, but I would guess 
there is a greater degree of portability in this area due to the 
very specific nature of the tests being processed. This 
information is found tabulated in table 6A. 

Table 6A* 
Hardware Configurations 

. Specialized Testing Programs 

Computer Used: 77" Scanner Used: 



Mainframes 


21% 


NCS 3000 


18% 


Minicomputers 


11 


NCS 7000-7003 


18 


Microcomputers 


29 


NCS 7005-7018 


14 


No indication 


39 


Scantron 


7 






Cognitronics 


4 






Other 


4 


Source: 




No indication 


36 


Commercial 


11 






Developed in-house 


71 


Plot Scores : 




Another institution 


7 


Yes 


18% 



No indication 11 

^Figures may not total 100% due to rounding error : 

Seventy-nine percent of the respondents indicated satisfaction 
with this type of software. 

Item Banking Software A large degree of interest was expressed 
Tn item banking software where we only had twenty-three 
questionnaires indicate they were doing this type of work . Table 
7 summarizes the methods of data input and the special features 
incorporated into these programs. 
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Table 7 
Item Banking Programs 



Feature 



Percent with this feature 



Accept data from 

Publisher tapes 
terminal screen 
word processing file 
text reader 
keypunch 

Upper/lower case 
Math, Chemical symbols 
Graphics capabilities 



4 
74 
30 
4 
4 

70 
30 
17 



n=23, all figures shown as percentages of n. 

As can be seen, the main method of data entry for item banking 
software is the terminal where questions are. typed using 
specialized programs for that purpose. The next largest method 
of data entry is the use of word processing programs where files 
are subsequently integrated into the item bank. A small 
percentage of users accept publisher 1 s data tapes or use a 
keypunch. The most innovative approach to data entry (and 
probably the most expensive) was the. use of a text reader where 
an entire page of information is read directly onto the system's 
disk files. 

These programs are resident on all three types of computers and, 
for the most part, were developed ih-house. 

Table 7A* 
Hardware Configurations 
Item Banking Programs 



Computer Used: 
Mainframes 
Minicomputers 
Microcomputers 
No indication 



35% 
13 
48 
4 



Source: 

Commercial _ 17% 

Developed in-house 74 
Another institution 9 



*Figures may not total 100% due to rounding error 

Expressed satisfaction for item banking software was considerably 
less chan for other types of_ software. Only 61% of users were 
happy with their programs. An interesting fact was that 26% of 
the respondents failed to indicate a degree of satisfaction. 
This was the only area where a significant number of people 
failed to rate their programs. 

Gradebook -^rog rants Another area of great interest that turned up 
in analyzing the surveys is that of gradebook programs . Twenty- 
eight of the sixty-four surveys indicated they were performing 
gradebook functions.. Features of these programs are listed in 
table 8. 



10 



ERIC 



9 



Gr 



Table 8 

Programs 



Feature t Perce 

Integrate with enrollment files 
Accumulate _ test grades 
Calculate final grades 
Differential weighting 



lis feature 



46% 
89 
75 
82 



h=28f all figures shown as percentages of n. 

Results on the second question oh the survey, "Does the program 
accumulate test grades?" were somewhat surprising. _ Only 89% of 
the respondents indicated a positive, response to this question . 
What about the other 11%? How does a gradebook program work if 
it doesn't accumulate grades? 

Miscellaneous Software Ih: order to see what was Being done other 
than in those areas listed, we asked you to comment on any other 
software that you were using. Almost everyone is using word 
processing and spreadsheet programs . Other areas are listed in 
tab '3 9. 

Table 9 

Miscellaneous Software- 



Data Analysis Programs 

SRSl 
SPSS 
BMDP 

misc developed in-house 



Teacher Evaluation Software 

all developed in-house 

Housing Questionnaires 
developed in-house 



CAI 

Typing. test 

Objective referenced testing 
On-line testing of English 
Usage 

Reader stats for holistic 
grading 

Item response theory modeling 



Scanning (general purpose) 

Microtest (NCS 3000) 
others developed in-house 

for NCS 3000 and 

Cognitronics 

Election Software 

developed ih-house 

Scheduling^ 

room assignment software 
developed in-house 
Communications software 

VTERM 

RNET 

PC3270 

customized data transfer 
software 



Psychological Testing 
MMPI 



PART III - THE "WISH LIST" Thirty-five of the sixtyrrfour 
respondents contributed to the "wish list"; Three areas seemed 
to be of greatest concern: test scoring programs, aradebook 
routines r and item banking. 

Test Scoring Programs Thirteen respondents desired increased 
test scoring capabilities. Of these/ 10 wished the programs to 
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run on microcomputers and three wished mairif rame-_based_ software. 
Six . of the 13 want to run the program on ah NCS 3000 scanner , 
while the rest mentioned other NCS equipment, a Scantrbn . 2100 
and a Cognitronics. Two were willing to pay up to $200.00 for 
the program; 3 offered up to $500.00? 1 indicated $1000,00; and 1 
offered "my first born". (He didn't say whether the first born 
was a boy or a girl!) Eight indicated they planned to develop 
the program in-house . 

Item Banking Programs Twelve institutions indicated a desire for 
increased capability in the area of item banking and test 
generation. Eight want the programs to be microcomputer based, 
the others opted for mainframe software* Only. five indicated an 
appropriate price to pay which ranged from $100.00 to $1000.00. 
Four surveys indicated the desired" programs would be developed 
in-house, while one was looking at a commercial product and one 
more was interested in a program f rem another institution. Six 
did not indicate a possible source. 

Gradebook Programs Nine surveys wish an improved gradebook 
program to be made avai lable . Three of these wanted 
microcomputer based programs, three wished mainframe, and three 
did not respond. Scanners mentioned to be used were the NCS 3000 
and NCS 7005. Only one indicated an appropriate cost, which was 
listed at "less than * $100.00. " The only source listed was one 
commercial. 

Other Desires Other desires expressed were for microcomputer- 
based routines for scoring psycho logical /vocational tests 
(primarily the SCII), scheduling and registration, teacher 
evaluations , adaptive testing , billing, and an interactive 
interpreter ( for programming) . Also requested was a mainframe- 
based program for scoring the ACT. Estimated costs varied 
greatly for the psych/voc tests where users wanted to pay 
anywhere from $250.00 to $750.00 for a program. Most wanted the 
program to run on the NCS 3000 and expected to purchase the 
program from a commercial source. 
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List of Available Software 

On the following pages are lists of all respondents by category 
who indicated they had software available that could be shared. 
In order to be included on this list, the "available to share" 
guest ion for each section had to be specifically marked "yes". 
Where costs, and options were indicated, they are listed. 
Remember r MSA does not specifically recommend any of the programs 
listed (we haven ■ 1 even tried them out ! ) , but simply lists those 
that members indicate are available. 

If you are looking for specific software, look first at the 
computer and scanner columns to find the. programs that utilize 
the hardware configuration you want . Next look at the options 
column to determine if the program would meet your needs, and 
then determine the source and contact the individual who can help 
you. 
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Features 



General Purpose Classroom Test Scoring _ 

Computer Scanner Source Cost Comments 



12 



1,2,3,4,7,8, 
1,2,3,4,6,9 
1,2,3,4,5,6, 
1,2,3,4,5,6, 
1,2,3,4,5,5, 
1,2^3^4,5,6^ 
none listed 
none .listed _ _ __ 
1,2,3,4^5,6,8,9,11 
1,2,3,4,7,9 
1,2,3,4,5,6, 
1,2,3,5,6,7, 
1,2,3*4,5*6, 
1,2,3,4,5,6, 
1,2,3,4,5,7, 
1*3,4,5,6, 7, 
1*2,3*5*6,8, 
1,2,3,4,5,6, 
1,2,3,4,5,6, 
1,2,3,4,5,6, 
1,2,3,4,5,6, 
1,2,3,4,5,6, 
1,2,4,6,8,9, 
1,2,3,4,6,7, 
none listed 
1,2,3,4,5,6, 
1,2,3,4,5,6, 
1,2,3,4,8,9 
1,2,3,4,5,6, 
1,2,3,4,5,6, 
3,5,12 

1,2,3,5-6,9,10,11 



9,11*12 __ 
2*8,9,11,12 
7*8,9,11,12 
7,9,12 



7*8,9,11 
8,9 

7*8*9*11*12 
7,9,11 
8,9,11,12 
8*9*10*11 
9,11,12 
7,8,9,11 
7,8,9,11 
7,9,10,11 
7*9,10,11 
7*8*9*12 
12 

9*11 

7,8,9,11 
7*9 

7*8*9*12 
7,8,9,11,12 



1 

3a 
3a 
2 

3a 

1 

1 

3a 
3a 
1 
1 

3a 
2 

3a 
7 

3a 
3a 

1 

3a 

2 

3 

3 c 

1 

1 

l*3a 

1 

1 

1 

1 

l*3a 
1 
1 
2 



1,2,3,4,5,6,7,8,9,11,12 

The following codes are used: 
Computer : 

1 mainframe 

2 minicomputer 

3a IBM PC or compatible 
3b Apple 

3c Other microcomputer 
Features : 



7018 
3000 
1400 
7001 
3000 
7001 
3881 

not clear 
7018 
7005+ 

3000,7010 
7005+ 
7005+ 
SCAN 

3000 

3000 

COGN 

COGN 

7005+ 

7005+ 

3000 

7001 

7005+ 
7 

7001 
7005*- 
7003 
7008 
COGN 
7001+ 
7005+ 
7018 



1 
2 
4 
6 
6 
7 
8 
9 

18 
21 
23 
24 
29 
32 
34 
35 
36 
41 
41 
42 
42 
44 
46 
49 
50 
51 
52 
54 
55 
56 
57 
59 
62 



25 



250 



9/87 



25 
350 

tape, shipping 



250 



40 



50 
50 
no fee 



Scanner : 
3000 



7001-18 

1400 

SCAN 

3881 

COGN 



NCS 3000 
NCS 7001-18 
Scantron 1400 
Scantron desktop 
IBM 3881 
Cognitronics 



1 


Item analysis 


7 


2 


Alpha list 


8 


3 


ID list 


9 


4 


Distribution table 


10 


5 


Scores to file 


11 


6 


Raw data to file 


12 



Source: 

Number refers to list of 



Subscores 

Differential weight's 
Reliability coefficients 
Collusion coefficients 
Graph or plot of scores 
Individual score report 



respondents attached* 
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t ajU^ntexest Testing 



13 



Tests— Scored Co m puter Scanner Source Cost Comments 

EPPS , SCI I , OP I , BP I , POI 3a 3 000, wo § 250 each 

SSHA, FIRO-B , MBTI , TJTS 3a 3000,'*oo g 250 each 

HERRMANN, CPI 3a 3000,^°° 6 250 each 

EPPS 3 a ? 20 

SCII,MMPI,C?I 2 7005+ 29 negotiable 

Multidimensional Personality 2 7005+ 29 negotiable 

Misc credentialing 2 SCAN 32 15,000 

Misc credentialing 3a SCAN 32 2,500 

MMPI ,MBTI 3a SCAN 37 

MBTI, Runner Scale 1 50 



The following codes are used: 

Computer: ______ Scanner: 

1 mainframe 3000 NCS 3000 

2 minicomputer 7001-18 NCS 7001-18 

3a IBM PC or compatible 1400 Scant ron 1400 

3b Apple SCAN Scant ron desktop 

3c Other microcomputer 3881 IBM 3881 

COGN Cognitronics 

Source: 

Number refers to list of respondents attached. 
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Placement Testing 



^n<^ M ^ 49 ^7 4M w ja% 

lesus beorea 


f * *w« «m 1 9 4* ^ 

Lomputer 


bcanner 


Source 


losl uonimenus 


MaCfl SKXxiS _ . 


J a. 


JUUU 


o 




MCwtaw alii KeauXug 


Jd 


jUUU 


0 


Q Q 

y y 


epAffi _ CPA'S TT 


-3 -a 

Jd 


JUUU 


o 


y y 


woop iingxisri 


ja 


JUUU 


o 


Q Q 

y y 


HDD 


• 


•p 


X o 




Ln^xxsn piaCClllCUL 


X 


jUUU/ /UlU 


4b J 


cct£s6 , snipping 


tmne - Taic*t"\ Cif hAiii 
X X Og f 11 LUU , Dnl , \Jt\x 






z y 


r\ A 4*f ^ 4 a W 1 4a 

nego t laoie 


ouonrora xhoi\ 


Z . 


/ UUDt 


^ Q 

z y 


negoiidDie 


na Cu 


Ja 


7 n n t 4- 


J ** 


Mawu/ to 


Jd 


JlU U Uv 




c( u 






jUUU 


J D 




Calif Achievement Tests 


any 


any 


41 




11C l» JL V JL LQll _ fiL 1 1 • J. CO L 


3b 


3000 

J U v w 






Math* English, TOEFL 


i 




50 




Nelson-Denny 


i 




50 




Nelson-Denny, Math, TSWE 


* 


7005+ 


52 




CEEB - 9 CLEF 


i 




55 




Math Placement , Michigan 


3a 


COGN 


56 




SAT, CEEB 


3a 


COGN 


56 




Math , language 


3a 


: 7005+ : 


60 




Language, Math 


2,3a 


3000,7018 


62 





The following codes are used: 



Computer: _ Scanner: 



1 


mainframe 


3000 


NCS 3000 


2 


minicomputer 


7001-18 


NCS 7001-18 


3a 


IBM PC or compatible 


1400 


Scantrbh 1400 


3b 


Apple 


SCAN 


Scantron desktop 


3c 


Other microcomputer 


3881 


IBM 3881 . 






COGN 


Cognitronics 



Source: 

Number refers to list of respondents attached. 
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Tests Scored 



Specialized Test Snoring 

Computer Scanner Source Cost Comments 





•3a 

Ja 


jUU U| Ifuv 


c 

O 




ACT complete system 


3a 


3000,»*<0O 


6 


250 


ACT,CLEP 


p 


? 


15 




SAT i TOEFL , MAT 


i 


7005+ 


19 




CLEP 


i 


3000,7010 


23 


tape, shipping 


ACT 


3a 


3000 


35 




ACT 


3c 


SCAN 


37 




MAT, ACT 


p 


• 7005+ 


52 




ACT, Kentucky Cbmpreherilive 


1 


7003 


54 




Henmon-Nelson , ITED,STEP-II 


2 


7018. 


52 




STEP- I I I , SCAT- I I , SCAT- I I I 


2 


7018 


62 





The following codes are used: 



Computer : 

1 
2 

3a 
3b 
3c 



mainframe 

minicomputer 

IBM PC or compatible 

Apple 

Other microcomputer 



Scanner : 
3000 



7001-18 

1400 

SCAN 

3881 

COGN 



Source: 

Number refers to list of respondents attached. 



NCS 3000 
NCS 7001-18 
Scantron 1400 
Scantron desktop 
IBM 3881 
Coghitronics 
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Item Banking, 



Feature s Computer Source Cost Comments 

2,5,6 3a 2 75-188 

2,3,5 2 6 

3 18 

2 1 27 ho fee 

2,5 3a 32 258 

2,3,5,6,7 3a 33 

1,2,5 1,2,3a 42 188 

2 3a 52 



The following codes are used: ~~ 

Computer: Scanner: 

1 mainframe 3888 NCS 3888 

2 minicomputer 7881-18 NCS 7881-18 
3a IBM PC or compatible 1488 Scantron 1400 

3b Apple SCAN Scantron desktop 

3c Other microcomputer 3881 IBM 3881 

COGN Cognitronics 

Features : 

1 publisher tape 5 upper/lower case 

2 terminal input 6 math,chem symbols 

3 word processing 7 graphics 

4 other input 

Source: 

Number refers to list of respondents attached. 
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Gradebodk 



Features 


Computer 


Scanner 


Source Cost Comments 


1,2,3,4 


3a 


3000 


2 


25-50 


1,2,3,4 


2 


7001 


5 




1,2,3,4 


3a 


7018 


18 




1,3,4 


1 


7005+ 


21 


500 Bendina 


3,4 


1 


7006 


27 


no fee 


1,3,4 


1 


COGN 


41 




1,3,4 . 


2 


7005+ 


42 


50 


1,2,3,4 


1 


7005+ 


59 


ho fee 


The following codes are used: 








Computer : 




Scanner : 






1 


mainframe 


3000 




NCS 3000 


2 


minicomputer 


7001' 


-18 


NCS 7001-18 


3a 


IBM PC or compatible 


1400 




Scantron 1400 


3b 


Apple 


SCAN 




Scantron. desktop 


3c 


Other microcomputer 


3881 




IBM 3881 






COGN 




Cognitronics 


Features : 










1 


differential weights 


3 


accumulate grades 


2 


integrate files 


4 


calculate grades 


Source: 










Number refers to list of 


respondents attached. 
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Application^ 



Computer— Scai 



Source^ 



Comments 



e&i 

General purpose scanning 
Faculty evaluations 
Faculty evaluations 
Holistic grading of essays 
(Questionnaire processing 
Faculty evaluations 
Faculty evaluations 
Faculty evaluations 
Housing questionnaires 
Survey data ^ 
Objective referenced test 
Score report program 
Faculty evaluation rating 
Faculty evaluation 
Election Ballots 
Room Assignments 
Faculty evaluation^ 
General purpose scanning 
Questionnaire responses 



3a 
3a 
3a 

i 

1 
j 

i 

2 

? 
p 

any 
any 
any 
1,3b 

3a 

•p 

i 

3a 

1 



3000 
3000 



7005+ 

7005+ 

7001+ 

7001+ 

7001+ 

any 

any 

any 

3000 

SCAN 

7005+ 

7003 

COGN 

7005+ 



2 
6 
6 
9 

12 
16 
21 
29 
34 
34 
34 
41 
41 
41 
46 
50 
52 
54 
56 
59 



25 
100 
250 



300 
negotiable 



no fee 



The following codes are used: 
Computer : 



Scanner : 



1 


mainframe 


3000 


Nes 3000 


2 


minicomputer 


7001-18 


NeS 7001-18 


3a 


IBM PC or compatible 


1400 


Scantron 1400 


3b 


Apple 


SCAN 


Scantron desktop 


3c 


Other microcomputer 


3881 


IBM 3881 






COGN 


Cognitronics 



Source: 

Number refers to list of respondents attached. 



List of Respondents by Institution 



1. Alabaaa, University of 
P.O. Drawer BE 
Tuscaloosa, AL 35487-1938 
Ray C. Saith 

2. Andrews University _ 
Berrien Springs* HI 49104 
Jeroee Thayer 

3. American Registry of 

Radiologic Technologists 
2600 Wayzata Blvd 
Minneapolis, MN 55405 
Jerry B Reid 

4. Arizona, State University 
University Testing Services 
302 Payne Ha 1 1 

Teepe, AZ 85287 
David J Krus 

5. Ball State University 
Coaput itig_ Services 
Muncie, IN 47306 

L. Henri ck^en 

6. Brigham Young University 
Testing Services 

265 HBB 

Pr ovo, Ut 84602 
Bud Hood 
(801) 378-6129 

7. Brookdale CoMunity College 
765 Me iwi a n Springs Rd. 

tlncrof t# m onrfa® 

Arnold Self Man 

8- Calgary, University of 

Office of Medical Education 
3300 Hospital Drive, N.M« 
Calgary, Alberta, Canada T2N-4N1 
Alain Chan 
Faculty of Medicine 

9. California, University of , Davis 
Teaching Resources Center 
Davis, Ca 95616 
Marina Estabrook 



10. Cal Poly _ 
Test Office 

San Luis Obispo, CA 93407 
Dr. Seorge C Stanton 

1 1 • Cal if State University , Fresno 
Office of Testing Services 
Fresno, CA 93740-0063 
Will lam P Stock 

I2i California State Univ., Fullerton 
Fuller ton, CA 92634 
Dr. John Gillis 

13. California State Univ. , Northridge 
181 ii BbrdfiSff St. 

Mbr thr idge , CA 9 i 330 
Phyllis Shaffer 

14. DuPage, College of 

22nd Street and Lambert Rd 
Slen Ellyn, lb 60137 
Gene Haliongren 

15. Eastern Illinois University 
Testing Services 
Charleston! IL 61920 
Herbert Bart ling 

16. Ferris State College 
Big Rapids, III 49307 
Fred Swart z 



17. Florida Atlantic University 
Testing and Evaluation 

500 NW 20th St. 
Boca Raton, FL 33431 
Dr. Lola Karl in 

18. Florida, University of 
0IR, 1012 Turlington 
Gainesville, FL 32611 
Sue H. Legg 

19. Georgia State University 
Box 692, University Plaza 
Atlanta, GA 30303 

Susan Ford Neel 
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20. Idaho 9 University of 

Student Counseling Center 
UCC 309 

Moscow* ID 83843 
Steve Saladin 

Si* Indiana University 

Blooeingtcn* IN 47405 
Clinton I Chase 



22- Indiana State Univarsi ty 
University Testing Office 
Reeve Hall* SS8 

Terre Haute, IN 47809 
Anna R Carson 

23- Illinois State University 
MAES — liS Julian Hail 
Nerval, It 61761 

Dr Elizabeth Harris 

S4w Kansas i University of 
116 Bailey 
Lawrence, ICS 66045 
Gary E Price 

25* Kent State University 
161 Rockwell Ball 
Kent, Wl 44242 
Wayne R Richards 

26. Massachusetts f Univ of 
DPC, Uhieore Building 
Aeherst, Mass 01003 
George E Comb 

27. Miaei* University of 
P.O. Box 249086 
Coral Gables, FL 33124 
Rod Gill is 

28. Michigan Dept of Licensing 
_ _ and Regulation 

P.O. Box 30018 

Lansing, MI 48909 
Rae Rafisdell 

29- Minnesota, University of 
192 Pi llsbury Dr. , S.E. 
Minneapolis* MN 55455 
ball is Perry 



30. Missouri* University of 
Missouri Testing & Eval. Service 
403 S. Sixth St. 

Columbia, MO 65211 
Harry R Snyder 

31. Montana State University 
Testing Service 
Bozesan, MT 59717 
Albert Suvak 

32. Natl. Board for Respiratory Care 
11015 W 75th Terrace 

Shawnee Mission, KS 66214 
Sally J Ktxen» Ph.D. 

33. NBOME 

1810 Elmaood 
Hilrette, IL 60091 
J. F. Smoley 

34. Nebraska* Univ of* Lincoln 
Teaching and Learning Center 
121 Benton 

Lincoln* NE 68588-0623 
Delivee L Wright 

35. Nebraska* Univ of* Omah^ 
60th and Dodge 

Omaha* NE 68182-0051 
Nick e Ewing 

36. New Mexico* University of 
Testing Division 
University College Bidg* Rm 2 
Albuquerque* NM 87131 

Craig Nobles 

37. New Mexico State University 
Counseling Center 

Las Graces, NM 88003 
John Duhing 

38. North Carolina State University 
Box 7209 _ 

B-21 Hillsbourough Bidg 
NCSU* Raleigh* NC 27695-7209 
Led Bucicmaster 

39. Sandra Robinson* UNO 
Counsel ing Center 

P.O. Box 8112 University Station 
Grand Forks* ND 58202 
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40. North Texas State Univ 
Counseling and Testing 
Denton* TX 76803 
Toe D Overton 
(817) 565-2741 

4t. Northern Illinois Univ 
Testing Service 
125 Altceld Hall 
DeKalb Illinois 6^115 
Lynn Owens 

42. Iowa State Mnivers* ;y 
32 Carver Hall 
A«c*» I A 50011 

Paul Lustgraaf 

43. NWHEL 

300 S. 6th 
Portland % OP 97204 
Dennis Deck 

44. Ohic State University 
226 Raeseyer Hall 

29 U_Hoo( ruff Ave 
Co luebus, ON 43210 
Ayres D* Costa 

45. Oklahoma* Univ, of 
555 Constitution 
IJoraan, OK 73037 
Helen Darks 

46. Oklahmoma State Univ 

Bureau of Tests and Measurements 
109 North Hurray Hall 
Stillwater* OK 74078-02*0 
Marilyn Ford 

47. Ottawa, Univ of 
School of Psychology 
Ottawa, Ontario , Canada 
KIN-6N5 

6. Sarrazin 

48. Pennsylvania State Univ 
University Testing Services 
211 Mitchell Bldg 
University Park, PA 16802 
Edeond Marks 



49. Pittsburgh, University of 
Office of Measurement and 

Evaluation 
Pittsburgh, PA 15260 
Carol Baker 

50. Sonoea State University 
1801 E Co tat i Avenue 
Rohnent Park, CA 94928 
tferald J A I ves 

51. South Dakota State University 
Radii 200 Adm Eldq 
Brookings, SD 57007 

Dean Hof land 



52. South Florida* Unive. sity of 
Evaluation arid Testing 

FAO 201 _ 
Taepa, FL 33620 
Harriet C Seligsdhri 

53. Southern Illinois University 
Learning Resources Service 
Carbondale, IL 62901 
Roberta Reeves 

54. Southwest Missouri State Univ 
901 S National 
Springfield, MO 65804 

Mark A Oglesby 

55. Texas, University of, Austin 
Measurement & Evaluation Ctr 
P.O. Box 7246 

Austin, TX 78713 
Bill Koch 

56. Texas A & M University 
Measurement and Research Svs 
261 Bizzell Hall West 
College Station, TX 77843 
Randy Nelson 

57 . Trenton State Co 1 lege 

• Hi 11 wood bakes CN4700 Green 10 
Trenton, NJ 08650 
A Daniel Phillips 
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58. Utah, University of 
Testing Center 

490 SSB 

Salt Lake City* UT 84112 
Judy Sch«idtH_evy 

59. Virgi nia tech 
5096 Herring 
Blacksburg, VA 24061 
Robert B Frary 

60 i Washington, Univ. of 
Educ Assess Ctr. 
PB - 30 

Seattle, fcla 98195 
Gerald M Gilieore 

61. Wayne State University 
Rft 343 MacKenzie Hall 
5050 Cass Ave _ 
Detroit, Ml 48203 
Thoeas 3 Uilhele 
C313I 577-3400 

62. Wisconsin, Univ of > Madison 
Testing & Evaluation Services 
1025 W Johnson, #366 
Madison, HI 53706 

Allah Cohen 
CSOBI 562-5863 

63. Wisconsin, Univ. oft Oshkosh 
Testing & Research Svs. 

800 Algoea 
Oshkosh, Hi 54901 
Tie H Hoyt 
(414) 424-1433 

64. Uyoaing, Univ. of 
Testing Center 
P.O. Box 3708_ 
Laraeie, WY 82071 
Ronald A Jackson 
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1996 Measurement Services 
Software Questionnaire 



This reproduction of the questionnaire can be used with 
the survey results to better help you to urider\stand the 
responses and explanations given* Questions should be 
directed to: 

Bud Wood 

265 HGB - BYU 

Prove, Ut 85602 



PART I - HARDWARE CONFIGURATION PRESENTLY USED 

tm Please indicate the computer (s) used by your office, whethe? 
actually located in your office or not and the percent of 
usage on each machine. 

X A. College or University Mainframe (Type 



X B. Minicomputer 

X C. Microcomputer (PC's) and then -heck 

( ) IBM pc or compatible 

( > Apple (including Macintosh) 

( > Other (specify- ) 

Please indicate the scanner (s) used by your office, whether 
ar tuaiiy located in your office or not and the percent of 
t^age on each machine. 



X 


A. 


NCS 2050 


X 


B. 


NCS 3000 


X 


e. 


NCS 7001, 7003, etc 


X 


D. 


NCS 7005, 7008, 7010, etc. 


X 


E. 


ScanTron standalone 


X 


F. 


ScanTron 520O 


X 


G. 


ScanTron System 9000 


X 


H. 


IBM 1230 


X 


I. 


Cognitronics 


X 


J. 


Other ( spec if y 
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PART II - SOFTWARE CORREliTLY USED 

A. General purpose classroom test scoring software (If hone, 
circle N/A and skip to next page.) 

1. Computer used if you have more than one 
(A - C in Part I r question 1,) 

2 . Scanner used if you have more than one 
(A - J in Part I r question 2.) 



3. Program name 



4. How did you obtain this- software.? (Circle A, B, or C) 
A. commercial 
source 



approx cost 



B. developed, in-house. 

available to share? Yes No 
if there is a fee, specify 

C. From other institution (specify) 



How essential is each 
Which of the following feature in your opinion? 
features are available 1. Essential 
in your test scoring 2. Useful, but not essential 

program(s)? (Please 3. Not Useful 

check) CIRCLE the appropriate number 

in the space to the right of 

each feature. 



Item analysis 

Alpha list of names and scores 

ID list of scores 

Score distrioution table 

Score data written* to- disk* or file 

Raw data (answers) written to disk 

or file 
Subscores available 
Differential weighting of items 
(i.e. , questions worth 2 poincs) 
Test reliability coefficient 
Collusion coefficients. 
Output plot or graph of scores 
Score report slip for each student 



_ ____ . _ . - _■»■»'— w k-rw*..*.**^ 

presently have? (please circle) yes 



i 2 


3 


i 2 


3 


i 2 


3 


i 2 


3 


i 2 


3 


i 2 


3 


i 2 


3 


i 2 


3 


i 2 


3 


1 2 


3 


1 2 


3 


1 2 


3 


system 


you 



no 



28 

ERIC 



EU Psychological /Vo : tional Int erest Test programs. Host of us 
provide some type of scoring service in support of 
counseling centers . Please indicate below the specific 
vocational and/or psychological inventories you computer 
score for counseling purposes* If you have different tests 
that are handled differently, please make a copy of this 
page and respond for each. (If none, circle N/A and skip to 
next page. ) 

1. Computer used if you have more than one 
(ft - C in Part I, question 1) 

2. Scanner used if you have more than one 
(ft - J in Part I, question.. 2.). ^ 

3. List the tests that are processed 



ftre scores plotted? yes no 

If so, is a graphics printer needed? yes no 

How did you obtain this- software? (Circle A, B, or C) 

ft. commercial 

source — 

approx cost 



B. developed in-house 

available to share? Yes No. 
if there is a fee, specify 



C. Prom other institution ( specify ) 



ftre you pleased with this software? yes no 



ERLC 
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C. Placement Testing. If _ydu. do piacemerit testing./ please 
complete this section of the ^estibririaire. If you do 
different kinds of testing in different ways - 9 please copy 
this section of the questionnaire and complete the 
information foi each method of testing, (If none circle N/A 
and skip to next page.) 

1. Computer used if you have mote than one 
(A - C in Part I f question 1) 

2. Scanner used if you have more than one * 
(A - J in Part I r question 2) 

3. List the tests that, are. processed_ 



ERLC 



4. Are scores plotted? yes no 

If sd r is a graphics printer needed? yes rib 

5. How did you obtain this software? (Circle A* B* or C) 
A. commercial 

source 



approx cost 



B . developed in-house 

available to share? Yes No 
if there is a fee r specify 



C. Prom other institution (specify) 



6. Are you pleased with this software? yes no 
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Special i^ed^ fceslr scor i ng software - If you have software 
developed to score a particular _test such _3_s_ACT, GED, etc, , 
please complete this section of _ the jjuestionnaire . Again , 
if you have multiple applications that are different f _ j>lease 
copy this page and complete the questionnaire for each 
unique application. . (If none f circle N/ft and skip to next 
page. ) 

1. Computer _use<l if you _ have more than one 
(ft • 6 in Part I f question 1 ) 

2. Scanner used if you have more than one 
(ft - j in Part I, question 2) 

3. List the tests that are processed 



4. Are scores pilot ted? yes no 

If sb f is a graphics printer heeded? yes ho 

5. How did you obtain this software? (eircie ft, B, or C) 
A. commercial 



source . _.. 
approx cost_ 



B. developed in-house 

available to share? Yes No 

if there is a fee, specify 



C. Prom other institution (specify y 



6. Are you pleased with this software? yes no 
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Item B anking and Test Generation Software, (If none? circle 
N/A and skip to next page.) 



1. Computer used if you have more than one 
(ft - C in Part I, question l) 

2. Where did you obtain this software? (Circle A, B, or C) 

A. commercial 

source 

approx cost 

B . developed in-house 

available to share? Yes No 

if there is. a- f ee^ specif y_ 

C. From other institution (specify) 

3. How are items entered into your item banks? (Circle 
all that apply) 

A. From publisher's tapes 

B. From terminal as part of item banking program 

C. From word processing text file 

D. Other (specify ) 

4. Does your program support (Circle all that apply) 

A. Upper/lower case 

B. Special math or chemistry symbols 

C. Graphics 

5. Are you pleased with this software? yes no 
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Grade Book Software - (If hone, circle N/ft and skip to next 
page.j 

1. Does your program integrate with enrollment files? 
yes ho 

2. Does the program, accumulate test grades for the entire 
term or semester? yes no 

3. Does it calculate final grades? yes. ho. 

4. Are options available to weight exams differently? 
yes ho 

5. Where did you obtain., this., softwane.?. (Circle. A, R, or. C). 

A. commercial 

source . 

approx cost 

B . developed ih-house = 

available to share? Yes No 

if there is a fee/ specify 

C. From other institution ( specify) 

6. Are you pleased with this software? yes no 
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Si Miscellaneous S oftware - Please Include below any software 
you are us ing that has -- not previously been covered. Copy 

this page and complete the information for each unique 

application you are reporting on. (If none, circle N/A and 
skip to Part III on next page.) 

1. Computer used if you have more than one 
(ft - 6 in Part I, question 1) 

2. Scanner used if you have more than one 
(ft - j in Part I, question 2 ) 

3. List the application 



4 . Special features 



How did you obtain this software? (Circle A, B, or C) 

*&• commercial 

source — 

approx cost , ~ 



B. developed in-house _ 

available^ to share? Yes No 
if there is a fee, specify 



G. From other institution (specify) 



Are you pleased with this software? . yes no 
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PART III - NEW SOFTWARE DESIRED 



Please indicate in this section , your "wish list 1 ' of software , 
the equipment you would like it to run on f and any special 

features desired. Next; list the cost you feel would, be 

reasonable to pay for such software and f inally , please indicate 
if you would prefer to buy the ^program or would prefer to develop 

it yourself. As before, please copy this page and provide the 

information requested on ail software for your,.. "wish list " . 

1. I would like software to 



2. Special features desired (if any) 



3. It should run on _ computer 
utilizing a(n) scanner. 

4. What would be a reasonable cost to pay for this program? 



5. I plan to buy/develop _ /self (please circle) the software. 
If buy, what is your anticipated source? 
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